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DISTRIBUTING CANDIDATE VECTORS BASED 
ON LOCAL MOTION COMPLEXITY 
FIELD OF THE INVENTION 

[0001] The present invention generally relates to video processing applications, and 
more particularly, to systems and methods for distributing candidate vectors based on local 
motacomplexl.,. 

5 DESCRIPTION OF RELATED ART 

[0002] For high quality video processing applications, motion estimation is typically 
used to perform a wide variety of beneficial functionalities, such as scan rate conversion, 
noise reduction, content encoding, and compression. In this regard, various motion 
estimation techniques have been developed to compensate for different characteristics of the 
10 video signal, different types of motion, and different objectives of the particular video 
processing application. 

[0003] These motion estimation techniques typically estimate motion for a block of 
pixels by evaluating a limited set of so-called candidate motion vectors. For each block of 
pixels, the motion estimation technique selects candidate vectors from a 3-D neighborhood 

IS by selecting firom among the detected motion vectors that are offset in the spatial and/or 
temporal domains. For example, candidate vectors may include one or more spatially offset 
motion vectors within the same frame and one or more temporally offset motion vectors 
within a previous or subsequent frame. Additional ''special" candidate vectors can also be 
used to take into account updated motion vectors or to compensate for zero motion, such as 

20 static text. In any event, the number of candidate vectors that are evaluated per block of 
pixels are typically fixed and predefined by the particular motion estimation technique. 
Every region, Aether easy or difficult, is evaluated using the same number of candidate 
vectors. 

[0004] Although fixing the number of candidate vectors sunpliiies the hardware and 
25 software implementation, these approaches can produce sub-optimal results. For example, 
in order to efSdently utilized limited computational resources and memory bandwidth, 
motion estimation should ideally odapt to the complexity of scene and the local content such 
that more emphasis is placed on picture regions tiiat are difficult to estimate, while less 
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emphasis is placed on regions fhat are easy to estimate (e.g. backgjrounds with uniform 
motion). It has also been shown that evaluating many candidate vectors m picture re^ons 
with uniform (converged) motion can introduce annoying distortion into the video signal, 
and therefore do more harm than good. By using a fixed number of predefined candidate 
S vectors, existing motion estimation techniques can produce an inefficient or undesirable 
allocation of limited resources. 

[0005] Therefore, in light of the foregoing problems, there is a need for systems and 
methods that efBciently distribute candidate motion vectors based on local motion 
complexity. These systems and methods would preferably be used to improve the 
10 allocation of limited resources in video processing applications that employ a motion 
estimation technique. 

SUMMARY OF THE INVENTION 

[0006] Embodunents of the present invention alleviate many of the foregoing problems 
by providing systems and mettiod for distributing candidate vectors based on local motion 

1 5 complexity. In one embodiment, a picture ftame is divided into a plurality of segments, 
where each segment includes a plurality of pixel blocks. The local motion complexity of 
each segment may then be measured by, for example, determimng a Sum-of-Absolute 
Differences (SAD) between pixel blocks of the firame and pixel blocks of an adjacent frame 
and summing the SAD values for the pixel blocks within each segment. The number of 

20 candidate vectors assigned to each block may then be determined based on the measured 
local motion complexity. 

[0007] Other embodiments provide mechanisms for assigning the mmiber of candidate 
. vectors to each pixel block. In one embodiment, for example, a distribution function for 
distributing candidate vectors may be determined using a maximum, minimum and average 

2S SAD value for the segments wittim the picture frame. The distribution fimction may also 
use predetermined values fi)r the maximum, minimum and average number of candidate 
vectors per block such that the distribution function achieves a desired distribution of 
candidate vectors. In tiiis way, the distribution function will distribute a higher number of 
candidate vector? to pbcel blocks witiiin segments having a higher SAD value (high motion 

30 complexity) and a lower number of candidate vectors to pixel blocks within segments 
having a lower SAD value (low motion complexity). This non-uniform distribution of the 
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number of candidate vectors can ensure that resources are re-directed to portions of the 
picture where such resources are needed. 

BRIEF DESCRIPTION OF THE DRAWING 

[0008] These and other features and advantages of the present invention will become 
5 more appaient to those skilled in the art from the following detailed description in 
conjunction wifli the appended drawings in which: 

[0009] Figure 1 illustrates an exemplary picture frame in accordance with one 
embodiment of the present invention; 

[0010] Figure 2 illustrates an exemplary method in flowchart form for distributing 
10 candidate vectors in accordance with one embodimmt of the present invention; and 

[0011] Figure 3 illustrates an exemplary platform that may be used m accordance with 
embodiments of the present invention. 

DETAILED EMBODIMENTS 

[0012] Embodiments of the present invention provide systems and methods for 
15 distributing candidate motion vectors based on local motion complexity. The following 
description is presented to enable a person sidlled in the art to make and use the invention. 
Descriptions of specific applications are provided only as examples. Various modifications^ 
substitutions and variations of the preferred embodiment will be readily apparent to those 
skilled in the art, and the generic principles defined herein may be applied to other 
20 embodimerits and applications without departing from the scope of the inventioa Thus, the 
present mvention is not intended to be limited to the described and illustrated embodiments, 
and should be accorded the widest scope consistent with the principles and features 

m 

disclosed herein. 

[0013] Embodunents of the present mvention may be used to efficiently allocate limited 
25 conq)utational resources to different portions of the picture frame without sacrificing 
motion estimation quality. In one embodiment, each picture frame is divided into a 
plurality of segments, where each segment mcludes a plurality of pbcel blocks. The number 
of candidate vectors evaluated for pixel blocks within each segment may then be 
determined based 'on an estimation of the relative motion complexity of each corresponding 
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segment. As such, the pixel blocks within the same segment are allocated the same number 
of candidate vectors, but pixel blocks within different segments could be allocated a 
different number of candidate vectors depending on the relative motion complexity of the 
different segments. This s^proach creates a non-uniform distribution of candidate motion 
S vectors (and associated computational resources used in their evaluation) for different 
groups of pixel blocks based on the relative motion complexity within each group. 

[00141 ^ oJ^d^r to estimate the relative motion complexity of each segment, the Sum-of- 
Absolute-Differences (SAD) between pixels of a frame and correspondmg pixels of an 
adjacent frame may be calculated, where the coiresponding pixels of the adjacent frame 
may be spatially aligned or spatially offset in accordance with a candidate motion vector. 
Because the SAD value in this case calculates the sum of the absolute value of differences 
of pbcels between blocks of adjacent frames, the SAD value of each segment provides the 
mismatch error of the blocks withm the segment, and therefore, can be used as an indirect 
measure of the motion complexity of each segment. If a segment has a high SAD value, 
then the segment likely has high motion complexity, and therefore the pixel blocks within 
the segment may benefit from a greater number of candidate motion vectors. If a segment 
has a low SAD value, then the segment likely has low motion complexity, and therefore, it 
may be moire efficient to allocate a lower number of candidate vectors to the pixel blocks 
within that segment By distributing a higher or lower number of candidate vectors based 
on the relative motion conq>lexity of each segment, limited computational resources can be 
non-uniformly distributed over different portions of the picture based on relative need. 

[0015] Referring to Figure I, an exemplary picture frame in accoidance with one 
embodiment of the present mvention is illustrated generally at 100. As illustrated, the 
exemplary frame 110 is divided into twenty-four picture segments 120, where each segment 
25 120 includes sixteen 8x8 pixel blocks 130. For purposes of clarity, the pixel blocks 130 are 
only illustrated for the first segment 120. It is understood that the exemplary frame of 
Figure 1 is provided for illustration purposes only and that other subdivisions of the picture 
frame are contemplated and embraced by the present invention. 

[0016] In order to estimate the motion complexity for each segment 120, a SAD value, is 
30 calculated between pixels blocks 1 30 of one frame 1 1 Oa and corresponding pixel blocks 1 30 
of an adjacent frame 1 10b. In this context, it is uiulerstood that the corresponding pixel 
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blocks 130 of the adjacent firame 1 10b could be spatially aligned with the pixel blocks 130 
of the current frame 1 10a. Alternatively, the corresponding pixel blocks 130 of the adjacent 
frame 110b could be spatially of&et with respect to pixel blocks 130 in the current frame 
1 10a based on an estimated motion vector that is determined using a limited set of candidate 
S motion vectors. Accordingly, for embodiments of die present invention, the correspondii^ 
pixel blocks 130 of adjacent frames 1 10b could be spatially aligned or spatially offset with 
resi^ct to the pixel blocks of the cunent frame 1 10a. 

[0017] Since each segment 120 includes a plurality of pixel blocks 1 30, the SAD values 
for each target segment 120 can be determined by summmg the SAD values for the 
10 individual pixel blocks 130 within the target segment 120. As such, the SAD value for each 
segm^t 120^piovides an estimate of mismatch error between the pixel blocks 130 within 
each segment 120, and therefore, an estimate of the local motion complexity of each 
segment 120. 

[0018] . Once the SAD value for each segment 120 has been determined, a distribution 
1 S function can be used to determine tiie number of candidate vectors to be evaluated for pixel 
blocks 130 within each segment 120 to produce a desired distribution of candidate vectors. 
For ^cample, suppose that there are sufficient resources to evaluate on average N_av 
number of candidates motion vectors fi>r all the blocks 130 within a frame 110. Furdier 
suppose tiiat it is desired to limit the number of candidate vectors to be evaluated to N_max 
20 and Nmin for the maximum and mimmum number of candidates per block 130, 
respectively. A distribution function for candidate vectors may then be determmed by 
applying the following rules: (1) evaluate N_min number of candidate vectors for pixel 
blocks 130 within segments that have the lowest match error (lowest SAD); (2) evaluate 
N_max number of candidate vectors for pixel blocks 130 within segments tiiat have the 
25 highest match-error (highest SAD); and (3) evaluate on average N_av number of candidate 
vectors per block for the blocks 130 within segments that have the average match-error. 
Based on these rules, the following exemplary equations would apply: 

N_min= a +b*SAD_mm+ c*SADjnin*SAD_min (1) 
30 Nmax = a + b*SADjnax + c*SAD_max*SAD_max (2) 

N_av = a + b*SAD_av + c*SAD_av*SAD_av (3) 
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[00191 fa equations (1) - (3), SAD^max, SADjnin and SAD_av correspond to the 
maximum, minimum and average SAD values, respectively, of all segments within the 
targeted frame. Furthermore, N.max, N_mm and N_av can be predetermined values for the 
maximum, minimum and average number of candidate vectors, respectively, that produce a 
S desired distribution of candidate vectors among the pbcel blocks. In a preferred 
embodiment, N_max is 9; Njmin is 4; and Njav is 6. 

[0020] If equations (1) - (3) above are solved simultaneously to determine the values of 
a, b and c, the following equation can be used to determine the number of candidate vectors 
to be evaluated within each segment based on the segment's corresponding SAD value: 

10 N_candi = a + b*SADi + c»SADi*SADi (4) 

,where N_candi is the number of candidate vector to be evaluated for pixel blocks within the 
i-th segment, and SADi is the SAD value for the i-th segment. By using equation (4) to 
determine the number of candidate vectors, the available number of motion vectors to 
evaluate can be distributed ov^ the various pbcel blocks according to the local motion 
IS complexity of each segment, i.e. the resources are spend at the locations where they are 
needed. The distribution function of equation (4) also allocates the number of candidate 
vectors such that distribution of candidate vectors conforms with the desired distribution 
(e.g., if SADi = SADjnax, then Njcandi ~ N_max; if SADi = SAD^min, then N^candj = 
N_min; and so forth). 

20 [0021] In comparison to schemes that use a fixed number of motion candidates per pixel 
blodc, the presented scheme can help to: (1) provide an improved motion estimation quality 
with the same total number of candidate vectors, and/or (2) provide a similar motion 
estimation quality with a fewer number of candidate vectors. For those skilled in the art, it 
is obvious that other sets of equations can be used and a different expression can be 

25 resolved to arrive at a (different) distribution fimctioa The presented set of distribution 
equations is just an example. Moreover, different or multiple measurements of local motion 
complexity can be taken into account For example, instead of using SAD values, 
embodiments of the present invention could use another estimate of mismatch error, such as 
Mean-Squared Error (MSB), without departing from the principles of the present invention. 

30 [0022] Referring to Figure 2, an exenq)lary metiiod in flowchart form for distributing 
candidate vectors in accordance with one embodiment of the present invention is illustrated 
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generally at 200. As illustrated, the exemplary method measures the local motion 
complexity at step 210 by, for example, measuring SAD or MSE values for a plurality of 
pixel blocks and summing the SAD or MSE values for a group of pixel blocks. A 
distribution function for determming the number of candidate vectors assigned to each pixel 
5 block is then determined at step 220. This process may involve developing a set of 
distribution equations, such as equations (1) - (4) above, that define a desired distribution of 
candidate vectors based on the measured local motion complexity. Hiese equations may 
include one or more anchor values, such a maximum, minhnum or average value for the 
measured motion complexity. 

1 0 [0023] At step 230, the number of candidate vectors to be evaluated for each pixel block 
is determined using, for example, the distribution fimction of equation (4) and the measured 
local complexity of the pixel blocks. Motion estimation for the applicable video frame may 
then be performed at step 240 using the number of candidate vectors assigned to each pixel 
block. It should be noted that the motion estimation technique should take into account the 

15 fact that a different number of candidate vectors may be assigned to different pixel blocks. 
For example, if the candidate vectors that could be evaluated include one or more spatial 
vectors, one or more temporal vectors, an update (modified motion) vector, a global motion 
vector, and a zero motion vector, the motion estimation technique may assign a priority 
order to each type of candidate vector. If the number of candidate vectors assigned to a 

20 pixel block is msufficient to evaluate all the possible candidate vectors, the motion 
estimation technique can then select fiom among the types of candidate vectors based on the 
vectors' relative priority. In tfiis way, fiie most relevant motion vectors would be evaluated 
for pixel blocks that are assigned a relatively small number of candidate vectors. 
Furthermore, if the number of candidate vectors are selected such that a smaller number of 

25 candidate vectors are assigned to pixel blocks wth relatively simple motion complexity, 
this limited set of motion vectors may be sufiBcient to compensate for the motion involved 
without sacrificing motion estimation quality. 

(00241 Referring to Figure 3, an exemplary platform that may be used in accordance 
with embodiments of the present invention is illustrated generally at 300. As illustmted, the 
30 exemplary platform includes a microprocessor 310 operably coupled to a memory system 
320 via a system bus 340. The memory system may comprise a random access memory, a 
hard drive, floppy drive, a compact disk, or other computer readable medium, that stores 
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con4>uter instructions for a motion estimation module 350 and video processing application 
ISO. The exemplary system also includes an I/O interface 330 that is coupled to the 
microprocessor 310 and the memory system 320 in order to enable the system to input and 
output video signals. 

S [002S] In operation, the video processing application 360 uses the motion estimation 
module 3S0 to process video signals received from the I/O interfoce 330 and to output the 
processed video signal to the I/O interfece 330. If the motion estunation module 350 is 
implemented in accordance with the principles described above, the exemplary system may 
be coniSgured to efiBciently utilize the limited processing resources of the microprocessor 

10 310 and the limited memory bandwidth of the memory system 350 without sacrificing 
motion estimation quality. Furthermore, if the motion estimation module 350 and/or video 
processing application 360 are implemented in hardware or combinations of hardware and 
software (instead of tiie software only implementation illustrated in Figure 3), other 
embodiments of the exemplary system would provide the system designer flexibility in 

1 5 designing the optimal configuration. 

[0026] While the present invention has been described with reference to exemplary 
embodiments, it will be readily apparent to those skilled in the art ibaX the invention is not 
limited to tiie disclosed and illustrated embodiments but, on tiie contrary, is intended to 
cover numerous other modifications, substitutions and variations and broad equivalent 
20 arrangements that are included within the scope of the following claims. 
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CLAIMS 

What is claimed is: 

1 . A method for distributing candidate motion vectois, the method comprising: 

dividing a picture frame (1 10) into a plurality of segments (120), each segment (120) 

con^rising a plurality of pixel blocks (130); 

measuring local motion complexity for each segment (120); and 

assigning a number of candidate motion vectors to pixel blocks (130) vnthin each 

segment (120) based on the measured local motion complexity. 

2. The method of claim 1» viieiein the step of measuring comprises: 

determining a sum-of-absolute differences between pixel blocks (130) of the picture 
frame (1 10a) and corresponding pixel blocks (130) of an adjacent frame (1 10b); and 

summing the measured sum-of^bsolute differences associated with of pixel blocks 
(130) within each segment (120). 

3. The method of claim 2, v^erein the step of assigning comprises using a distribution 
function configmed to assign the number of candidate vectors based on the measured local 
motion complexity of each segment (120). 

4. The method of claim 3, wherem the distribution frmction is based on a maximum, 
minimum and average of the measured sum*of-absolute differences of the segments. 

5. The method of claim 4, wherein the distribution friction is frirther based on 
predetermined values for a tn^Trimmrij mimmiitn and average number of candidate vectors 
per block* 

6. The method of claim 1, frirther comprising performing motion estimation on the pixel 
blocks (130) using the number of candidate vectors assigned to each pixel block (1 30). 
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7. A system for distributing candidate vectors^ the system comprising: 

means for dividing a picture frame (1 1 0) into a plurality of segments (120), each 

segment (120) comprising a plurality of pixel blocks (130); 

means for measuring local motion complexity for each segment (120); and 
means assigning a number of candidate motion vectors to pixel blocks (130) within 

each segment (120) based on the measured local motion complexity. 

8. The system of claim 7, wherein the means for measiiring comprises: 

means for determining a sum-of-absolute differences between pixel blocks (1 30) of 
the picture frame (1 10a) and corresponding pixel blocks (130) of an adjacent frame (1 10b); 
and 

means for summing the measured sum-of-absolute differences associated with of 
pbcel blocks (130) within each segment (120). 

9. The system of claun 8, whereui the means for assigning uses a distribution function 
configured to assign the number of candidate vectors based on the measured local motion 
complexity of each segment (12). 

10. The system of claim 9, wherein the distribution function is based on a maximum, 
minimum and average of the measured sum-of-absolute differences of the segments.. 

1 1 . The system of claim 10, wherein the distribution friction is fiirther based on 
predetermined values for a maximum, minimum and average number of candidate vectors 

I per block. 

■ 

12. The system of claim 7, frirth^ comprising means for performing motion estimation on 
the pixel blocks (130) usmg the number of candidate vectors assigned to each pixel block 
(130). 
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ABSTRACT 

A system and method for distributing candidate motion vectors divides a picture frame 
(1 10) into a plurality of segments (120), where each segment (120) includes a plurality of 
pixel blocks (130). The local motion complexity of each segment may then be measured by 
determining a Sum-of-Absolute Differences (SAD) between pixel blocks (130) of the frame 
(1 10a) and pixel blocks (130) of an adacent frame (1 10b) and summing the SAD values for 
the pixel blocks (130) within each segment (120). The maximum, mimmum and average 
SAD value for the segments may be used to define a distribution function that distributes 
candidate vectors to conform with a desired maximum, minimum and average number of 
candidate vectors per block. This distribution function is then used to detemine the number 
of candidate vectors to assign to pixel blocks (130) based on the measured SAD value of the 
associated segment (120). 



Fig.l 



CoDv orovided by USPTO from the IFW Image Database on 11/24/2004 



BEST AVAILABLE COPY 






'J 



: ( 



\ . • • 

• 



Copy provided by USPTO from the IFW Image Database on 11/24«04 




Co«v orovided bv USPTO f ram tho IFW tmago Database on 




'g' "W.J ' 





n I r- Ti* 



• ■ 









f 





I 



Copy provided by USPTO from the IFW Image Database on 1 1/24/2004 



